Non-markovian process modelling with Echo State Networks
نویسندگان
چکیده
Reservoir Computing (RC) is a relatively recent architecture for using recurrent neural networks. It has shown interesting performance in a wide range of tasks despite the simple training rules. We use it here in a logistic regression (LogR) framework. Considering non-Markovian time series with a hidden variable, we show that RC can be used to estimate the transition probabilities at each time step and also to estimate the hidden variable. We also show that it outperforms classical LogR on this task. Finally, it can be used to extract invariants from a stochastic series.
منابع مشابه
ANALYSIS OF FINITE BUFFER RENEWAL INPUT QUEUE WITH BALKING AND MARKOVIAN SERVICE PROCESS
This paper presents the analysis of a renewal input finite buffer queue wherein the customers can decide either to join the queue with a probability or balk. The service process is Markovian service process ($MSP$) governed by an underlying $m$-state Markov chain. Employing the supplementary variable and imbedded Markov chain techniques, the steady-state system length distributions at pre...
متن کاملProcess Algebraic Non-product-forms
A generalization of the Reversed Compound Agent Theorem of Markovian process algebra is derived that yields separable, but non-product-form solutions for collections of interacting processes such as arise in multi-class queueing networks with Processor Sharing servers. It is based on an analysis of the minimal cycles in the state space of a multi-agent cooperation, which can be simply identifie...
متن کاملA Partially Observable Markovian Maintenance Process with Continuous Cost Functions
In this paper a two-state Markovian maintenance process where the true state is unknown will be considered. The operating cost per period is a continuous random variable which depends on the state of the process. If investigation cost is incurred at the beginning of any period, the system wit I be returned to the "in-control" state instantaneously. This problem is solved using the average crite...
متن کاملDissertation an Echo State Model of Non-markovian Reinforcement Learning
OF DISSERTATION AN ECHO STATE MODEL OF NON-MARKOVIAN REINFORCEMENT LEARNING There exists a growing need for intelligent, autonomous control strategies that operate in real-world domains. Theoretically the state-action space must exhibit the Markov property in order for reinforcement learning to be applicable. Empirical evidence, however, suggests that reinforcement learning also applies to doma...
متن کاملA Markovian characterization of redundancy in echo state networks by PCA
Richness of dynamics is a desirable feature of Echo State Networks (ESNs) limited by a known high redundancy of state units activations. We show how this feature is mainly influenced by the Markovian state space organization of ESNs through a Principal Component Analysis (PCA) of the reservoir space. Relevances of principal components are coherent with Markovianity, whose role is further enligh...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2009